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Preface 


This  paper  discusses  design  considerations  for  a  prototype  distributed  group  support 
system  as  well  as  an  initial  assessment  of  its  demonstrated  functionality.  The  system 
I  created  enabled  participants  in  a  group  process  to  interact  with  each  other  across  time 

and  distance.  Pilot  studies  were  performed  to  measure  user  acceptance  and  system 
effectiveness  and  usability. 

This  work  supports  the  Armstrong  Laboratory,  Logistics  Research  Division,  Acquisition 
I"  Logistics  Branch's  (AL/HRGA)  ongoing  work  in  the  area  of  developing  and  demonstrating 

various  integrated  tools  and  techniques  to  aid  in  implementing  Integrated  Product 
Development  (IPD)  and  support  the  in-house  capability  to  perform  research  and  development 
in  design  decision  support,  information  technology  and  information  integration  for  weapon 
system  requirements  development  and  product  design  (work  unit  number  1710-00-18). 
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Design  Considerations  for  FrankTalk 
A  Distributed  Group  Support  System 


Introduction 

Most  research  into  the  development  and  use  of  computers  to  support  group  work  has  focused 
on  systems  designed  to  work  primarily  in  a  same-time,  same-place  environment.  Systems  of 
this  type  generally  consist  of  networked  computers  set  up  in  a  "decision  room"  environment 
[Heminger,  1989].  With  this  type  of  system,  groups  can  be  united  to  work  through  a  structured 
process  to  achieve  group  goals.  Such  a  facility,  the  Group  Research  Laboratory  for  Logistics 
(GRLL),  has  been  created  at  Armstrong  Laboratory's  Logistics  Research  Division  at  Wright- 
Patterson  Air  Force  Base  (AFB),  Ohio.  Based  on  our  experiences  with  the  use  of  the  GRLL, 
we  came  to  believe  that  there  is  a  need  for  the  capabilities  of  the  GRLL,  even  when  the  group 
cannot  meet  face-to-face.  There  are  times  and  situations  where  a  group  of  people  needs  to 
work  together  on  a  problem  when  separated  by  time  and/or  distance. 

We  call  systems  that  provide  this  support  a  Distributed  Group  Support  System  (DGSS).  Based 
on  the  expressed  interest  in  a  DGSS,  a  prototype  system  was  designed  and  created  at  the 
Logistics  Research  Division  of  Armstrong  Laboratory.  This  paper  discusses  design 
considerations  for  this  system  as  well  as  an  initial  assessment  of  its  demonstrated 
functionality. 


Background 

The  Director  of  Armstrong  Laboratory  was  among  those  who  recognized  a  need  for  this  type  of 
system.  As  the  director  of  a  laboratory  whose  members  were  geographically  dispersed,  he 
supported  the  development  of  a  system  to  provide  a  more  structured  form  of  communication. 
He  saw  that  such  a  system  would  help  groups  go  beyond  their  current  use  of  electronic  mail 
and  enable  them  to  undertake  and  achieve  their  collaborative  goals. 

The  director  wanted  to  be  able  to  electronically  “invite”  up  to  three  dozen  participants  at  one 
time  (from  a  potential  pool  of  all  Armstrong  Laboratory  personnel)  to  participate  in  what  would 
be  an  electronic  virtual  meeting.  Based  on  the  director's  needs  and  our  knowledge  of  group 
support  systems,  we  decided  that  we  wanted  a  system  that  would; 

•  allow  a  person  to  call  a  meeting  and  specify  a  list  of  participants; 

•  enable  the  person  calling  the  meeting  to  pose  a  problem  or  issue  to  the  invited 
participants; 

•  allow  participation  from  multiple,  dispersed  participants; 

•  allow  participants  to  generate  ideas,  discuss  those  ideas,  and  do  rank  order  voting  on 
lists  of  items; 

•  support  anonymous  idea  generation  and  voting;  and 

•  conclude  by  selecting  the  three  or  four  best  answers  to  the  problem  for  further  study. 
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The  director  stated  that  this  system  had  the  potential  to  “enhance  the  overall  sense  of  value  of 
all  the  people  in  his  organization,”  and  improve  the  Total  Quality  (TQ)  of  Armstrong  Laboratory. 


Systems  Currently  Available 

Existing  products  which  performed  all  the  required  functions  (inviting  participants, 
brainstorming,  consolidating  ideas,  and  ranking  ideas)  were  reviewed,  but  they  were  limited  to 
a  single  Local  Area  Network  (LAN).  In  that  regard,  none  of  the  products  satisfied  the 
“geographically  dispersed”  requirement  of  the  director.  Another  limitation  of  the  evaluated 
products  was  the  need  to  have  a  homogenous  computing  environment  consisting  entirely  of 
IBM®-type  personal  computers  (PCs),  entirely  of  Macintosh®  computers,  or  entirely  of  X- 
windows®-based  machines.  There  were  no  products  identified  that  would  operate  across 
dissimilar  computer  platforms  within  a  LAN  environment;  therefore,  whichever  platform  was 
chosen  (PC,  Macintosh,  or  X-Windows),  the  product  would  not  be  available  to  those  laboratory 
personnel  with  different  computers  (without  the  purchase  of  additional  hardware). 

An  alternative  to  a  LAN-based  product  was  a  software  program  running  on  a  host  computer 
with  access  to  the  Defense  Data  Network  (DDN).  With  such  an  arrangement,  anyone  with 
access  to  the  DDN  would  be  able  to  reach  the  DGSS  and  take  part  in  the  group  work.  Since 
nearly  all  Armstrong  Laboratory  personnel  have  access  to  the  DDN,  the  geographic 
requirement  is  met.  The  VAX  computer  cluster  at  the  Logistics  Research  Division  is  connected 
to  the  DDN,  thus  it  could  serve  as  a  host  for  this  type  of  system.  However,  software  that  met 
the  laboratory  director's  needs  which  could  be  supported  in  this  VAX  environment  was  not 
found;  it  would  have  to  be  developed  by  Armstrong  Laboratory  personnel. 

Creating  such  a  system  for  the  VAX  would  be  both  time  consuming  and  expensive.  Therefore, 
we  decided  that  before  undertaking  the  effort  necessary  to  create  a  VAX-based  system,  we 
would  create  a  LAN-based  system  and  investigate  it  for  utility,  consequently  reducing  the  initial 
commitment  to  the  concept,  in  case  the  idea  was  shown  not  to  have  merit.  A  PC-LAN-based 
DGSS  would  be  installed  on  the  Armstrong  Laboratory  Human  Resources  Directorate  LAN  and 
tested.  Depending  on  the  level  of  user  acceptance,  the  PC-LAN-based  system  could  be 
extended  to  work  as  a  wide-area  network  (WAN)  application;  or  if  more  appropriate,  a  host 
based  application  would  be  developed  by  the  Logistics  Research  Division  for  use  on  the  DDN. 

In  January,  1992,  we  installed  a  PC-LAN-based  system  on  the  Human  Resources  Directorate 
LAN  at  Brooks  AFB,  Texas.  However,  at  the  time  of  the  test,  the  LAN  administration  policy  did 
not  allow  multi-user  software  (e.g.,  there  was  no  provision  for  shared  user  directories,  which 
effectively  negated  the  usefulness  of  groupware).  For  purposes  of  testing,  however,  the  LAN 
administrator  set  up  a  five-user  system  for  one  week,  after  which  the  software  was  removed 
from  the  LAN.  The  test  indicated  that  the  software  would  not  be  effective  over  the  Human 
Resources  Directorate  LAN  as  that  LAN  was  then  configured,  so  efforts  were  directed  toward 
development  of  the  host-based  system  for  use  on  the  DDN.  That  system  came  to  be  called 
FrankTalk. 


Creation  of  FrankTalk 

FrankTalk,  a  distributed  group  support  system  (DGSS)  designed  to  run  on  VAX  VMS,  was 
developed  between  January  and  March,  1 992.  It  was  created  by  modifying  available  public- 
domain,  bulletin-board  software  distributed  by  Digital  Equipment  Corporation  (DEC),  the  maker 
of  VAX  computers.  Using  this  software  as  a  starting  point,  we  were  able  to  rapidly  develop  a 
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prototype  system  at  minimal  cost.  The  basic  functionality  of  the  bulletin-board  software 
provided  the  capability  to  create  structured  group  processes  and  to  collect  and  display 
participant  comments.  We  modified  it  to  allow  removal  of  user  IDs  from  user  comments,  thus 
providing  anonymity.  In  addition,  we  added  a  menu-driven  user  interface,  along  with  a  basic 
voting  module.  Since  the  system  was  developed  as  a  research  tool,  a  logging  function  was 
also  included  to  capture  usage  data. 

As  created,  FrankTalk  met  our  initial  design  requirements. 

•  It  allowed  the  meeting  leader  to  set  up  a  meeting  and  to  specify  the  roster  of 
participants. 

•  It  enabled  the  person  calling  the  meeting  to  pose  a  problem  or  issue  for  the 
participants. 

•  It  supported  input  from  multiple,  dispersed  participants. 

•  It  allowed  idea  generation,  idea  discussion,  and  rank-order  voting. 

•  It  supported  anonymity  for  the  idea  generation  and  voting  modules. 

•  It  allowed  the  meeting  leader  and  participants  to  select  the  group's  choice  of  best 
answers. 

Once  FrankTalk  was  installed,  it  could  be  accessed  in  any  of  the  ways  the  VAX  computer 
would  normally  be  accessed.  For  users  on  whose  VAX  the  software  was  installed,  it  was 
available  as  an  option  from  their  regular  accounts.  For  those  out  of  the  local  area,  access  was 
available  either  via  modem  connection  or  by  way  of  the  DDN.  For  all  interested  users  of  the 
system  at  the  Logistics  Research  Division,  an  account  was  made  available  with  the  approval  of 
their  supervisors. 

For  a  typical  session,  a  leader  would  set  up  a  meeting,  including  the  agenda,  the  list  of  invited 
participants,  and  the  timetable  for  the  meeting.  Participants  would  be  notified  of  the  meeting 
and  would  be  expected  to  participate.  To  participate,  they  would  log  into  the  system,  select 
the  appropriate  meeting,  and  take  part  in  the  active  process.  As  the  meeting  unfolded, 
participants  would  be  able  to  see  their  input,  along  with  that  of  the  other  participants.  The 
meeting  leader  would  move  the  agenda  along  as  each  process  was  completed.  At  the 
conclusion,  the  meeting  in  its  entirety  would  be  available  by  computer,  as  well  as  in  the  written 
reports  that  could  be  created.  For  further  information  on  the  use  of  FrankTalk,  see  the  User 
Manual  in  the  appendix. 


Pilot  Testing  FrankTalk 

Two  pilot  studies  were  undertaken,  both  to  determine  the  basic  functionality  of  the  system  and 
to  get  initial  responses  from  the  users.  A  small  pilot  was  set  up  on  the  VAX  at  the  Logistics 
Research  Division  at  Wright-Patterson  AFB,  Ohio  (where  FrankTalk  was  developed)  and  was 
restricted  to  the  people  in  that  division.  A  larger  pilot  with  a  larger  user  base  was  run  using  the 
Human  Resources  Directorate  VAX  at  Brooks  AFB,  Texas.  This  larger  pilot  encompassed 
members  of  the  Armstrong  Laboratory  headquarters,  its  directorates,  and  their  divisions, 
including  divisions  in  Arizona  and  Ohio  (see  Figure  1).  The  accompanying  discussion  and 
figures  describe  the  usage  of  FrankTalk  in  both  pilots. 
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The  Small  Pilot  Installation 


FrankTalk  was  installed  for  the  small  pilot  study  on  March  25,  1992,  and  restricted  to  a  single 
division.  Approximately  70  persons  had  access  to  the  system,  although  with  turnover,  a  total 
of  approximately  80  persons  could  have  accessed  the  system  over  the  first  year  of  operation. 
Availability  of  the  system  was  announced  at  a  division-wide  meeting  on  March  27.  Details  on 
the  usage  of  this  system  during  its  first  year  of  operation  are  shown  in  the  accompanying 
Figures. 

Figure  2  shows  the  number  of  persons  accessing  the  system  each  day  over  the  period  from 
March  1992  to  February  1993. 
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Figure  2.  The  number  of  users  accessing  the  smail  piiot  system  per  day 
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Initially,  the  system  was  set  up  as  an  electronic  suggestion  box.  In  a  division  call  on  May  8, 
1992,  the  division  chief  announced  that  the  electronic  suggestion  box  would  replace  the  former 
paper  system  and  encouraged  all  division  members  to  use  it.  Over  the  course  of  the  year, 
other  groups  asked  that  private  conferences  be  created.  Each  of  these  conferences  were 
accessible  only  to  the  groups  who  requested  them  and,  of  course,  to  the  computer  system 
administrator.  The  activity  peaks  in  September  and  December  1992,  resulted  when  a  branch 
chief  encouraged  the  use  of  the  system  by  the  members  of  her  branch. 

Figure  3  shows  the  distribution  of  system  usage  among  the  people  involved  in  the  small  pilot 
study.  The  graph  indicates  two  trends:  1)  some  users  initially  tried  the  system,  accessed  it  a 
few  times,  but  then  quit;  and  2)  some  users  accessed  the  system  many  times  (e.g.,  about  half 
the  people  accessed  the  system  more  than  10  times  each  during  the  test  period,  and  10 
percent  accessed  the  system  more  than  100  times). 
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Figure  3.  The  number  of  FrankTalk  accesses  per  user  during  small  pilot  study 
The  Large  Pilot  Installation 

On  18  May,  1992,  we  installed  FrankTalk  for  the  large  pilot  study  on  the  VAX  at  Brooks  AFB 
and  trained  the  people  there  to  use  it.  The  large  pilot  study  had  a  pool  of  approximately  400 
potential  FrankTalk  users  and  was  directly  available  to  most  of  the  divisions  of  the  Human 
Resources  Directorate,  as  well  as  to  other  directorates  of  the  laboratory.  The  Human 
Resources  Director  asked  that  all  personnel  within  the  directorate  building  be  given  access  to 
the  system.  Approximately  40  percent  of  the  personnel  in  the  building  obtained  accounts  on 
the  VAX  and  attended  the  training  sessions.  In  addition,  the  Director  of  Armstrong  Laboratory 
asked  that  all  laboratory  Directors,  Deputy  Directors,  and  Division  Chiefs  receive  accounts  and 
training.  Approximately  50  percent  of  these  attended  the  training  session.  Again,  the  system 
was  originally  established  as  an  electronic  suggestion  box  which  all  of  the  users  could  read 
and  post  messages  on.  Additionally,  a  conference  restricted  to  Directors  and  a  conference 
restricted  to  Directors  and  Division  Chiefs  were  created. 

We  then  traveled  to  Williams  AFB,  Arizona,  to  train  the  Division  Chief  and  ensure  he  could 
access  FrankTalk.  Next,  we  returned  to  Brooks  AFB  to  train  administrators  of  the  system  and 
users  who  had  missed  the  first  training  session.  A  final  trip  was  made  to  visit  all  Directors  and 
Division  Chiefs  in  their  own  offices  to  ensure  they  were  able  to  connect  to  the  system. 

FrankTalk  training  at  Brooks  AFB  was  provided  in  a  computer  training  room  using  dedicated 
VAX  terminals.  These  training  sessions  provided  an  orientation  to  the  concept  of  using 
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FrankTalk,  the  ability  to  make  sure  everyone  who  wanted  one  had  a  valid  VAX  user  ID,  and  an 
opportunity  to  ensure  distribution  of  the  FrankTalk  user's  guides.  Everyone  was  given  a 
chance  to  practice  using  the  menu-oriented  FrankTalk  interface. 

The  laboratory  Director  proposed,  as  the  first  formal  conference,  a  brainstorming  session  on 
possible  uses  of  the  group  support  system.  Responses  were  requested  from  the  end  of 
December,  1992,  until  mid-January,  1993. 

Usage  was  logged  beginning  in  July,  1992,  and  is  shown  for  the  period  July,  1992  to  March, 
1993  (see  Figures  4  and  5).  More  than  120  persons  have  accessed  the  system  at  least  once 
since  logging  was  started  in  July  (after  the  training  sessions)  and  more  than  80  persons  have 
accessed  the  system  at  least  twice.  Figure  4  shows  the  total  number  of  users  of  the  system 
each  day.  Figure  5  shows  the  distribution  of  the  system  usage  among  the  users  of  the  system. 
Usage  within  the  large  pilot  study  can  be  compared  with  that  of  the  small  pilot  study  shown  in 
Figures  2  and  3. 

These  data  suggest  that  people  tended  toward  one  of  two  responses  to  the  use  of  FrankTalk. 
About  25  percent  of  the  users  in  each  pilot  accessed  FrankTalk  only  once,  suggesting  that 
they  did  not  find  it  to  be  particularly  useful.  On  the  other  hand,  about  50  percent  of  the  users 
in  each  pilot  accessed  FrankTalk  nine  or  more  times,  suggesting  that  this  group  may  have 
found  FrankTalk  to  have  some  value.  Of  course,  other  explanations  for  the  distribution  of  the 
data  may  exist.  Additional  data  is  required  to  support  our  tentative  conclusion. 


Dates  of  the  large  pilot:  July  1992  -  March  1993 

Figure  4.  The  number  of  users  accessing  FrankTalk  per  day  during  the  large  pilot  study 
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Number  Of  times  that  FrankTalk  was  accessed 

Figure  5.  Percent  of  users  accessing  FrankTalk  a  given  number  of  times 

Few  messages  were  posted  to  the  conferences  restricted  to  directors  and  division  chiefs.  Only 
seven  messages  were  posted  for  the  entire  month  of  February,  and  five  of  those  were  simply 
practice  messages.  Nevertheless,  some  personnel  continued  to  access  the  system. 
Throughout  the  pilot,  an  average  of  approximately  seven  users  per  day  accessed  the  system. 
During  the  first  months  of  1993,  the  average  was  approximately  six  users  per  day,  out  of  a 
potential  population  of  400  users. 


Discussion 

Since  these  studies  were  pilot  studies,  our  interest  was  in  finding  out  if  the  system  could 
support  the  efforts  of  groups  working  together  on  tasks,  although  working  from  different  places 
at  different  times.  To  do  this,  we  looked  at  three  issues:  effectiveness,  usability,  and  user 
acceptance.  Similar  criteria  have  been  used  as  critical  success  factors  for  assessing  the 
success  of  other  group-oriented  computer  technologies  [Heminger,  1989].  In  that  case, 
Heminger  used  efficiency  instead  of  usability.  However,  usability  can  be  seen  as  a  part  of 
efficiency  because  a  system  which  is  difficult  to  use  will  be  seen  as  inefficient  in  terms  of  user 
energy  and  attention  to  make  it  work.  At  this  pilot  stage  of  testing,  our  interest  was  focused  on 
the  usability  issue. 

Effectiveness 

The  question  to  ask  about  effectiveness  is  whether  the  system  was  able  to  fulfill  its  intended 
purpose.  In  this  case,  the  question  was  whether  the  system  could  be  successfully  used  to 
support  a  group  that  was  undertaking  a  goal-oriented  task.  Each  pilot  group  was  asked  to  use 
FrankTalk  as  a  collection  box  for  suggestions  on  various  topics.  In  each  pilot,  ideas  were 
submitted  anonymously  from  multiple  users  at  various  locations,  thus  fulfilling  a  basic 
expectation  for  the  system.  In  addition,  the  participants  in  the  small  pilot  used  FrankTalk  for  a 
more  complex  process  involving  group  generation  of  ideas,  organization  of  the  ideas  into  a 
series  of  alternatives,  and  rank-ordering  of  the  alternatives.  In  all  three  cases,  the  users  were 
able  to  use  the  system  to  carry  out  the  assigned  tasks.  Thus,  the  system  demonstrated  that  it 
could  meet  the  effectiveness  criterion. 
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Usability 

Data  collected  indicates  that  many  users  had  problems  related  to  the  software  interface  as  well 
as  to  the  type  of  terminal  or  terminal  emulator  they  used.  The  interface  is  based  on,  and 
therefore  looks  and  performs  like,  a  VMS  application,  a  nongraphical,  command-based 
interface  in  which  the  user  must  either  remember  and  then  enter  commands  directly  or  enter 
the  number  of  an  item  from  a  list.  In  comparison  with  today's  graphical  interfaces,  this 
environment  is  less  user-friendly. 

Menus  are  displayed  to  the  users  at  all  times,  but  at  the  lowest  levels  of  FrankTalk,  except  in 
the  editor,  a  list  of  appropriate  commands  are  listed  at  the  bottom  of  the  screen.  However, 
while  a  user  adds  a  comment,  the  editor  does  not  display  the  commands  necessary  to  save 
the  comment  and  exit  from  the  editor.  Many  users  found  pressing  <Ctrl>+"Z",  then  typing 
"EXIT",  as  a  way  to  send  a  comment  to  be  less  than  obvious. 

An  additional  problem  was  that  the  computer  users  at  the  Human  Resources  Directorate  at 
Brooks  AFB  had  recently  transitioned  from  the  VAX  to  their  PC  LAN.  Nearly  all  the  PC  LAN 
users  had  given  up  their  VAX  accounts  and  now  relied  entirely  on  their  PCs.  Re-establishing 
their  VAX  accounts  required  many  of  them  to  learn  how  to  use  a  terminal  emulator  on  their  PC. 
Not  all  PC  keyboards  mapped  exactly  to  the  keys  found  on  the  dedicated  VAX  terminals  used 
in  the  training  sessions,  causing  problems  for  some  users. 

Also,  unrelated  to  the  FrankTalk  program  itself,  the  users  had  to  remember  to  use  two 
separate  passwords  before  they  could  log  into  FrankTalk.  The  VAX  itself  required  the  use  of 
computer  generated  passwords  comprised  of  random,  non-word  sequences.  In  addition,  a 
separate  password  was  needed  to  log  into  the  LAN  that  provided  the  connection  to  the  VAX. 
Many  users  were  annoyed  by  this  inconvenience,  and  a  number  of  them  probably  gave  up 
using  the  system  when  they  forgot  their  passwords. 

Slow  system  response  was  also  indicated  as  a  barrier  to  FrankTalk  usability  by  those  who 
accessed  FrankTalk  across  the  DDN.  Users  reported  that  their  TELNET^  session  would 
occasionally  time-out  before  being  able  to  login  to  the  remote  host,  thus  forcing  them  to  re¬ 
initiate  the  process  of  logging  into  the  system.  Once  into  the  system,  the  response  time  over 
the  DDN  often  became  prohibitively  long.  At  times,  the  response  time  was  so  long  that  tens  of 
seconds  would  pass  from  the  time  a  key  was  pressed  until  the  result  was  shown  on  the 
screen.  Under  such  conditions,  the  users  frequently  became  discouraged  and  gave  up  using 
the  system. 

User  Acceptance 

As  Heminger  [1989]  points  out,  a  system  that  is  not  accepted  by  its  intended  users  is  not  used, 
and  thus  produces  no  value  to  the  organization.  Although,  this  seems  to  be  an  obvious 
tautology,  it  is  one  that  has  often  been  missed  by  system  designers  over  the  years.  It  is 
important  to  find  out  whether  people  will  actually  use  the  system  to  do  their  jobs.  From  the 
usage  data,  it  is  clear  that  while  some  users  accessed  the  system  more  regularly  than  others, 
(50  percent  of  each  pilot  group  used  the  system  nine  or  more  times),  the  system  did  not  gain 
the  widespread  regular  use  that  would  lead  to  routine  adoption  of  the  technology.  With  such 
infusions  of  technology,  a  critical  mass  of  system  users  is  necessary  for  its  continued  use. 


^TELNET  is  an  Internet  standard  protocol  for  communication.  The  purpose  of  the  TELNET  protocol 
is  to  provide  a  fairly  general  communications  facility.  Its  primary  goal  is  to  allow  a  standard  method  of 
interfacing  terminal  devices  and  terminal-oriented  processes  to  each  other.  For  more  information  on 
TELNET,  see  the  Advanced  Research  Projects  Adminstration  Request  for  Comments:  854,  Telnet  Protocol 
Specification. 
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Without  a  critical  mass  of  users,  people  are  less  likely  to  continue  to  use  a  system,  since  they 
will  find  that  they  must  still  rely  on  additional  channels  to  complete  their  work.  Figure  2  shows 
that  in  the  small  pilot,  the  users  demonstrated  low  frequency  of  access  from  the  beginning  of 
the  trial.  In  contrast,  the  large  pilot  study  participants  (Figure  4)  show  a  somewhat  stronger 
initial  use  of  the  system,  followed  by  a  tapering  off  of  use.  In  both  trials,  there  was  a  mid-trial 
surge  of  activity,  caused  by  specific  urging  from  management  to  use  the  system.  However,  for 
both  studies,  the  flurry  of  activity  quickly  tapered  off,  continuing  the  pattern  of  decreasing  use. 

The  low  turnout  for  training,  coupled  with  the  fact  that  the  use  patterns  show  a  relatively  low 
use  throughout  the  pilot,  suggests  that  the  potential  users  did  not  accept  the  prototype  system 
as  a  viable  alternative  for  carrying  out  group  work. 

Some  of  the  problems  stemmed  from  the  platform  of  the  system.  The  dated  user  interface 
was  found  to  be  difficult  for  users.  It  presented  a  plain,  and  to  some,  unattractive  screen  with 
a  command-driven  or  menu-driven  interface.  It  required  remembering  various  commands, 
which  if  forgotten,  could  strand  a  user  in  the  middle  of  a  session.  These  problems  were 
probably  accentuated  for  the  users  in  the  large  pilot  study  by  the  fact  that  they  had  recently 
moved  from  a  VMS-based  system  to  a  DOS-based  LAN.  Thus,  using  FrankTalk  meant  going 
backwards  in  technology. 

An  additional  problem  stemmed  from  the  configuration  of  the  DDN,  which  made  timely  use  of 
FrankTalk  difficult.  A  lesson  here  is  that,  although  the  configuration  of  the  network  is  outside 
the  control  of  the  researchers,  it  nonetheless  had  important  consequences  for  user 
acceptance.  In  a  computer  environment,  any  weak  link  can  degrade  or  interrupt  the  overall 
performance  of  the  system.  When  working  in  a  network  environment,  it  is  important  to  pay 
attention  to  all  of  the  components  that  can  affect  performance  and  acceptance. 

Applegate  [1991]  states  that  the  presence  of  an  effective  management  sponsor  is  positively 
correlated  with  adoption  and  ownership  of  the  system.  In  this  case,  even  though  there  were 
management  sponsors  for  both  pilot  tests,  and  even  though  the  pilots  were  run  within  divisions 
of  Armstrong  Lab,  the  same  lab  where  the  system  was  created,  it  did  not  meet  with  wide  user 
acceptance,  suggesting  that  the  issues  of  user  interface  and  system  response  time  are 
powerful  deterrents  to  the  acceptance  of  a  new  technology. 


Conclusions 

This  study  was  undertaken  to  develop  and  to  test  a  DGSS.  The  results  of  the  study  show  that 
we  did  create  a  workable  DGSS,  albeit  one  that  did  not  meet  with  strong  user  acceptance. 
FrankTalk  is  capable  of  being  used  by  a  dispersed  group  to  generate  ideas,  organize  the 
ideas,  and  vote  on  a  list  of  alternatives.  However,  its  lack  of  user  acceptance  suggests  that  it 
will  not  be  a  successful  product  in  its  current  state. 

Its  lack  of  acceptance  appears  to  be  at  least  partly  based  on  usability  issues,  particularly  a 
dated  user  interface,  a  difficulty  in  connecting  to  the  VAX  computer  on  which  the  program 
resided,  and  long  response  times  for  those  that  were  connected  to  the  system  via  the  DDN.  A 
Graphical  User  Interface  (GUI)  was  a  common  request  from  FrankTalk  users. 

Fifty  percent  of  the  users  from  each  study  accessed  FrankTalk  at  least  nine  times,  suggesting 
that  people  were  willing  to  give  the  system  a  try,  but  that  it  did  not  provide  sufficient  usability  to 
be  successfully  integrated  into  the  working  environment. 

Based  on  the  results  of  this  investigation,  it  is  clear  that  to  be  successful,  a  DGSS  must  not 
only  provide  specific  structuring  of  group  processes,  but  it  must  provide  structuring  within  a 
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framework  that  meets  the  usability  requirements  of  the  intended  users.  Since  user 
expectations  tend  to  be  a  moving  target  based  on  available  technology,  it  will  be  important  to 
carefully  consider  the  current  state  of  user  interfaces  when  creating  the  next  generation  of  this 
product. 
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ACRONYMS 


AFB 

Air  Force  Base 

DDN 

DEC 

DGSS 

Defense  Data  Network 

Digital  Equipment  Corporation 

Distributed  Group  Support  System 

GRLL 

GUI 

Group  Research  Laboratory  for  Logistics 
Graphical  User  Interface 

LAN 

Local  Area  Network 

PC 

Personal  Computer 

TQ 

Total  Quality 

WAN 

Wide-Area  Network 
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When  the  meeting  leader  decides  that  voting  is  complete,  the  technographer  puts  conference,  get  in  touch  with  your  local  FrankTalk  focal  point  or  Capt  Kennon 

the  meeting  in  REVIEW  phase.  In  this  phase,  participants  may  see  the  results  of  the  Moen  (DSN  785-8363;  Commercial  513-255-8363). 
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